Experiments with Reinforcement Learning in Environments with Progressive Difficulty

This paper introduces Progressive Reinforcement Learning, which augments standard Q-Learning with a mechanism for transferring experience gained in one problem to new but related problems. In this approach, an agent acquires experience of operating in a simple domain through experimentation. It then engages in a period of introspection, during which it rationalises the experience gained and formulates symbolic knowledge describing how to behave in that simple domain. When subsequently experimenting in a more complex but related domain, it is guided by this knowledge until it gains direct experience. A test environment with 15 mazes, arranged in order of difficulty, is described. Experiments in this environment demonstrate the benefit of Progressive RL relative to a basic RL approach in which each puzzle is solved from scratch. 1